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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )H Responsive to communication(s) filed on 20 December 2004 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) (3 Claim(s) 3 and 5-16 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) E3 Claim(s) 7, 12, and 14 is/are allowed. 

6) EI Claim(s) 3,5,6,8-11.13,15 and 16 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) K The drawing(s) filed on 20 December 2004 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d)or (f). 
bM AH b)D Some * c)U None of: 

1 M Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Art Unit: 2655 

DETAILED ACTION 
Response to Amendment 

1. In response to the office action from 9/20/2004, the applicant has submitted an 
amendment, filed 12/20/2004, amending the abstract and Claims 3, 5, and 6-10, while canceling 
Claims 1, 2, and 4, adding Claims 11-16, and arguing to traverse the art rejection based on the 
limitation regarding a comparison between the upper and lower limit of a reaction time and three 
modes of third-party assistance involvement (Amendment, Page 13). Applicants arguments have 
been fully considered, however the previous rejection is maintained due to the reasons listed 
below in the response to arguments. 

2. Based on the amendments to the drawings, the examiner has withdrawn the previous 
objections directed towards minor informalities. 

3. Based on the amendment of Claim 10, the examiner has withdrawn the previous 35 
U.S.C 101 rejection. 

Response to Arguments 



4. Applicant's arguments have been fully considered but they are not persuasive for the 
following reasons: 
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With respect to Claim 3, the applicant argues that Marx et al (U.S. Patent: 6,173,266) . 
fails to teach a comparison between the upper limit and lower limit of the reaction time in the 
ordinary interaction (Amendment, Page 13), however Marx teaches a timeout condition (upper 
limit) used to initialize a call transfer to an operator (Col. 8, Lines 20-31; Col. 9, Lines 52-65; 
previous office action, page 8). 

In response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., the 
comparison between an upper and a lower limit is not claimed; only that an upper or lower limit 
is used to initialize assistance from a third party is claimed) are not recited in the rejected 
claim(s). Although the claims are interpreted in light of the specification, limitations from the 
specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 18 1, 26 
USPQ2d 1057 (Fed. Cir. 1993). 

Thus, since Marx teaches a timeout threshold for determining whether it is necessary to 
transfer a user to an operator, the rejection of Claim 3 is maintained. 

With respect to Claims 5 and 15-16, the applicant argues that Bohacek et al (U.S. Patent: 
6,41 1,687), Marx, and Davis et al (U.S. Patent: 5,583,922) fails to teach a progress of interaction 
in including three modes of operator involvement (Amendment, Page 13), however the examiner 
notes that Davis teaches progressive modes of interaction operation including updating user data 
at an operator terminal, parallel input of data, and telephone call transfer for direct interaction 
(Col. 8, Lines 19-35, and Col 11, Lines 48-58). The applicant also argues that the prior art of 
record is not properly combinable, however provides no reasons as to why (Amendment, Page 
13). Thus, the below rejection is considered to be a sufficient response to these arguments. 
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The applicant provides no specific arguments directed towards the rejection of Claims 9 
and 10. Thus, the below rejection is considered to be a sufficient response. 

The dependent claims are argued as further limiting their associated rejected parent 
claims (Amendment, Page 13), and therefore also remain rejected. 

Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1 (a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1(2) of such treaty in the English language. 

6. Claim 9 is rejected under 35 U.S.C. 102(e) as being anticipated by Bohacek et al (U.S. 
Patent: 6,411687), 

With respect to Claim 9, Bohacek discloses: 

A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col 3, Lines 60-65); 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col. 3, Line 48- Col 4, Line 8)\ 
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A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col. 4, Lines 
40-48, and Col 3, Lines 17-30)- 

An interaction engine for extracting contents of a response corresponding to the voice 
information by referring to a knowledge database, and creating a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col. 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts)', and 

A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
smoothly conducted, allows a third-party user to participate in interaction between the user and 
the interaction engine from another terminal as a helper (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col 3, 
Lines 17-30, and Col 4, Lines 40-48). 

Claim Rejections -.55 USC §103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
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having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. Claims 3, 6, and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Bohacek et al (U.S. Patent: 6,41 1,687) in view of Marx et al (U.S. Patent: 6,173,266). 

With respect to Claim 3, Bohacek discloses: 

A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col. 3, Lines 60-65)', 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col. 3, Line 48- Col. 4, Line 8)\ 

A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col. 4, Lines 
40-48, and Col 3, Lines 17-30); 

An interaction engine for extracting contents of a response corresponding to the voice 
information by referring to a knowledge database, and creating a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col. 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts); and 

A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
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smoothly conducted, allows a third-party user to participate in interaction between the user and 
the interaction engine from another terminal as a helper (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col. 3, 
Lines 17-30, and Col. 4, Lines 40-48). 

Bohacek does not specifically suggest the ability to transfer a call to an operator when a 
response reaction time limit is exceeded, however, Marx recites: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on an average reaction time from a response of the interaction 
engine to a reaction of the user, and in a case where the average reaction time exceeds a first 
threshold value or in a case where the average reaction time is below a second threshold value, 
allows a third-party user to participate in the interaction between the user and the interaction 
engine from another terminal as a helper (timeout condition used to initialize a call transfer to an 
operator, Col. 8, Lines 20-31, and Col 9, Lines 52-65). 

Bohacek and Marx are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to combine the use of a timeout condition to initialize a 
call transfer to an operator as taught by Marx with the interactive voice response system capable 
of transferring a caller to an operator upon detecting a predetermined level of frustration based 
on speech data as taught by Bohacek in order to provide a further means of assisting an IVR 
system user in accomplishing a desired call result when a lack of response indicates possible user 
confusion. Therefore, it would have been obvious to combine Marx with Bohacek for the benefit 
of providing proper user assistance in response to user confusion indicated by a lack of response. 



Application/Control Number: 09/940,522 Page 8 

Art Unit: 2655 

With respect to Claim 6, Bohacek discloses: 

The interaction engine further includes an interaction history information storage part for 
recording interaction history on a user basis, and a helper selection part for selecting the third- 
party user that is considered to be most familiar with the contents of the interaction from the 
interaction history as a helper, and the helper most appropriate for contents of the voice 
information is selected (previous customer data used to route a call to the most appropriate 
operator, Col 4, Lines 24-39, and Fig. 1). 

With respect to Claim 8, Bohacek recites: 

Interaction history display part for displaying the interaction history stored in the 
interaction history information storage part to a third-party helper user, and a helper instruction 
part for receiving a help instruction from the third-party helper user, wherein when the help 
instruction part receives the help instruction from the third-party helper user, the voice 
information mediation part enables the interaction between the third-party helper user and the 
user to be conducted, and when a degree of help of the third-party helper user exceeds a 
predetermined threshold value in interaction between the third-party helper user and the user, the 
interaction engine interacts only with the third-party helper user (transferring interaction control 
to an appropriate operator when an annoyance threshold is exceeded, displaying customer data, 
and previous customer data, Col 3, Lines 17-30, and Col 4, Lines 30-39). 

9. Claim 5, 11, 13, and 15-16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bohacek et al in view of Marx et al, and further in view of Davis et al (U.S. Patent: 5,583,922). 
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With respect to Claims 5 and 15, Bohacek in view of Marx teaches the interactive voice 
response system and method capable of transferring a caller to an operator based upon a 
frustration level, recognition engine error amount, or a timeout condition, as applied to Claim 3. 
Bohacek in view of Marx does not specifically suggest the ability to determine interaction 
progress through interaction time and access amounts in order to initialize parallel input 
operator/user interaction, however Davis recites: 

The contents of interaction with the user is displayed to the third-party user and the 
contents of interaction is updated by the third-party user, parallel input in which the third-party 
user conducts an input in parallel with the user, to switching in which the third-party user 
directly interacts with the user (user information transferred to an operator and direct 
communication through data or voice, Col 11, Lines 48-58). 

Bohacek, Marx, and Davis are analogous art because they are from a similar field of 
endeavor in interactive voice response systems. Thus, it would have been obvious to a person of 
ordinary skill in the art, at the time of invention, to combine the use of a parallel user/operator 
input as taught by Davis with the interactive voice response system and method capable of 
transferring a caller to an operator based upon a frustration level, recognition engine error 
amount, or a timeout condition as taught by Bohacek in view of Marx in order to provide a 
means for a user to maintain some system control while being aided in information entry by an 
operator to ensure data accuracy in the condition that recognition or timeout errors occur. 
Therefore, it would have been obvious to combine Davis with Bohacek in view of Marx for the 
benefit of providing parallel input user assistance. 
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Claim 11 contains subject matter similar to Claim 6, and thus, is rejected for the same 
reasons. 

Claim 13 contains subject matter similar to Claim 8, and thus, is rejected for the same 
reasons. 

Claim 16 contains subject matter similar to Claims 5 and 10, and thus, is rejected for the 
same reasons. 

10. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bohacek. 
With respect to Claim 10, Bohacek teaches the interactive voice response system and 

method capable of transferring a caller to an operator upon detecting a predetermined level of 
frustration based on speech data, as applied to Claim 1 . Although Bohacek does not specifically 
suggest method storage on a computer readable medium, the examiner takes official notice that it 
would have been obvious to one of ordinary skill in the art, at the time of invention, to store the 
IVR method taught by Bohacek on a computer readable medium to increase method 
compatibility and usability by providing a means for method use with multiple computer 
systems. 

Allowable Subject Matter 

1 1 . Claims 7, 12, and 14 are allowed. 



12. 



The following is a statement of reasons for the indication of allowable subject matter: 
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With respect to Claim 7, the prior art of record does not explicitly teach nor fairly 
suggests: the ability of an operator to assume complete control of an voice interaction engine in 
an interactive voice response system when it is detected that only an operator's voice continues 
for a predetermined time period during an error-initiated operator interaction session, in 
combination with a help request notification used for informing an operator of a user's failure to 
respond to an interactive voice system (error) and allowing the operator to voluntarily interact 
with the user in response to such a notification. Further the prior art of record also does not teach 
or fairly suggest the above features in combination with the aforementioned voice response 
system, wherein the voice response system utilizes a voice information mediation portion for 
performing the interaction error detection based on response time and sound quality thresholds to 
decide whether to transfer a user to third party operator assistance. 

Claims 12 and 14 further limit allowable claim 7, and thus are also considered allowable. 

Conclusion 

13. THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
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CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

14. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Stuart et al (U.S. Patent: 6,243,684)- teaches an interactive voice response system 
having varying levels of operator interaction. 

15. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632 
and email is James.Wozniak@uspto.gov. The examiner can normally be reached on Mondays- 
Fridays, 8:30-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wayne Young can be reached at (571) 272-7582. The fax/phone number for the 
Technology Center 2600 where this application is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the technology center receptionist whose telephone number is (703) 306- 



0377. 



James S. Wozniak 
5/25/2005 




DAVID L. OMETZ 
PRIMARY EXAMINER 



